MIT OpenCourseWare 
http://ocw.mit.edu 



6.854J / 18.415J Advanced Algorithms 
Fall 2008 



For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms . 



18.415/6.854 Advanced Algorithms November 3, 2008 

Lecture 16: Approximation Algorithms 

Michel X. Goemans 



1 Approximation Algorithms 

Many optimizations problems arising in practice are NP hard. Under the widely accepted conjecture 
that P ^ NP, we cannot compute efficiently and exactly an optimal solution for all possible instances 
of these problems. Several approaches have been used to deal with this intractability. On one 
hand, dynamic programming, branch and bound, and implicit enumeration algorithms always find 
an optimal solution by navigating the space of feasible solutions in a more efficient way than an 
exhaustive search, but their running time is not guaranteed to be polynomial in the input's size. On 
the other hand, heuristic algorithms provide a sub-optimal solution to the problem, but their running 
time is polynomial in the size of the input problem. In this lecture we will focus on approximation 
algorithms, which are heuristics that always find a solution whose objective value is guaranteed to 
be within a certain factor of the optimum solution. 

Definition 1 (Approximation Algorithm) LetV be a minimization (resp. maximization) prob- 
lem with instances / 6 I. An a- approximation factor for a > 1 (resp. a < 1) algorithm for V is 
an algorithm A whose running time is polynomial in the size of the given instance I, and outputs a 
feasible solution of cost ca such that ca < ct ■ OPTj (resp. ca > ct ■ OPTj ), where OPTj is the cost 
of the optimal solution for instance I . 

In this lecture, we will discuss three general techniques of designing approximation algorithms 
for NP-hard problems: 

1. Using optimal value in the analysis without explicitly knowing it. 

2. Linear programming relaxation and rounding. 

3. Primal-dual technique. 

2 A 3/2- Approximation Algorithm for the Metric TSP 

The Traveling Salesman Problem is one of the most extensively studied problems in combinatorial 
optimization. In the metric version of the problem, an instance is a complete undirected graph 
G = (V, E) and c : E — > M + , where c satisfies the metric property: c(u, v) = c(v, u) for all u, v G V, 
and the triangle inequality, c(u,v) < c(u,w) + c(w,v), for all u,v,w G V. The objective is to find 
tour, that is a cycle visiting every vertex exactly once (also called a tour) minimum cost . 
A | approximation algorithm for this problem by Christofides [1] is as follows. 

1. Find a minimum spanning tree T of G. 

2. Compute a minimum cost perfect matching M on the set of odd-degree vertices V oc id Q T. 

3. Find an Eulcrian tour C' (a cycle visiting all the edges exactly once) in M U T. 

4. Output the tour C that visits the vertices of G in the order of their first appearance in the C' . 
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Figure 1: Execution of Christofidcs' algorithm on an instance. The first figure shows a minimum 
cost spanning tree. The second figure shows the addition of a minimum cost matching on odd degree 
vertices in the tree, and the third figure shows a cycle obtained after "shortcutting" an Eulerian 
tour in the previous graph, starting from vertex 1. 

Theorem 1 The above algorithm is a 3/ '2- approximation algorithm for the metric TSP. 

Proof: It is clear that all steps in the algorithm can be implemented in polynomial time. The 
minimum spanning tree can be found using a greedy algorithm, and the minimum cost matching 
for V dd can be found in polynomial time using the ellipsoid algorithm, as discussed in one of the 
previous lectures (or by a purely combinatorial algorithm also based on the linear program we 
discussed). Note that c(T) < OPT, because the optimal tour without an edge becomes a tree. Also, 
c(M) < OPT/2. To see this, consider any optimal tour, and then short-cut it to get a cycle visiting 
only vertices in V dd with cost at most OPT. Since the cycle induces two matchings consisting of 
alternating edges, at least one of them will have cost at most OPT/2. From this, the total cost of the 
Eulerian cycle, an upper bound of the cost of the algorithm, is at most OPT + OPT/ 2 = 3/2 • OPT. 

□ 

Note that in the analysis of the algorithm, we used the value of OPT even without explicitly 
computing it exactly, or getting a lower bound on it. Figure 1 shows an instance of the metric TSP, 
and the execution of the algorithm on this instance. 

A few remarks: 

• The above analysis for the algorithm is tight, i.e. Ve > there is an instance I such that the 
algorithm returns a solution which is 3/2 — e times the optimal solution. 

• Currently, no algorithm with an approximation factor better than 3/2 is known for metric 
TSP. 

• TSP is known to be MAX-SNP hard [5] even for the case when distances are either 1 or 2. 
Also, Papadimitriou and Vempala [4] have proved that a 1.01 approximation algorithm for the 
metric TSP will imply P = NP. 

3 Designing Approximation Algorithms via Relaxations 

One of the most important paradigms in the design of approximation algorithms are relaxations. 
Consider the following (hard) minimization problem. 

min f(x) 
s.t. x e S. 
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One approach to solve this problem is to extend S to a bigger set P D S where the same problem 
is easier to solve. Namely, we extend the function / to a function g : P — > R satisfying g{x) = 
f(x), Vx € S (or g(x) < f(x)). If this condition holds, then 

min/(x) > mmg(x), 

which gives a lower bound for the value of the optimal solution. Therefore, if an algorithm gives a 
solution i*£S which satisfies f(x*) < a min ie p g(x), then this is an a- approximation algorithm 
for the problem. 

For example, many combinatorial optimization problems can be expressed as 

T 

mine x 

s.t. Ax = b, 

x e {0,1}". 

A natural relaxation is to replace the integrality constraint Xi e {0,1} by the linear constraint 
< Xi < 1, we obtain the LP relaxation of the integer program above. 

min c T x 
s.t. Ax = b, 

0< Xi < 1, Vi = 1, . . . , n. 

In some cases, the polytopc corresponding to the LP relaxation has all integral extreme points. 
In such cases, it is sufficient to solve the LP relaxation to solve the original problem exactly. But 
this is not true in general. 



3.1 LP Relaxation for the Vertex Cover Problem 

Given an undirected graph G = (V,E), a vertex cover of G is a collection of vertices C C V such 
that all edges e = (u, v) in E satisfy C D {u, v} ^ 0. The Vertex Cover problem on an instance 
G = (V,E),c : E — > R + is to find a cover C of G of minimum cost c(C) = X)«ec c ( v )- This is known 
to be an NP-hard problem. 

A natural formulation using integer variables and linear constraints is the following. We define a 
variable x u € {0, 1} which takes value 1 if it is in the vertex cover, otherwise. Then the following 
is an integer programming formulation for the vertex cover problem. 



min c v x v (la) 

v£V 

s.t. x u + x v > 1, V(u, v) £ E, (lb) 

x u G {0,1}, MueV. (lc) 



The LP relaxation for the vertex cover problem is 



(2a) 

V(u,v)eE, (2b) 
Vit e V. (2c) 

Note that we removed the x u < 1 constraints, since if x u > 1 we can change it to x u — 1 without 
increasing the cost, and still have a feasible solution. 



mm > CyXy 



s.t. x u H - x^, ^ 1, 
x u > 0, 
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Xx = 1/2 




x 2 = 1/2 x 3 = 1/2 

Figure 2: An example where the LP relaxation for the Vertex Cover does not have an integer optimal 
solution. 



The LP relaxation does not necessarily have an optimal integral solution in general. For example, 
consider the graph given in Figure 3.1 with all costs equal to 1. The optimal solution for this instance 
has cost OPT = 2 , but the optimal solution for the LP relaxation has cost LP = 3/2, as shown in 
the figure. What this example shows is not only that LP < OPT in general, but also an interesting 
fact about the strength of this relaxation. Suppose that we are going to use LP as a lower bound 
on OPT in order to prove an approximation guarantee. As we will see in the next subsection, we 
will be able to find a cover C with cost at most 2LP. Therefore, we can say 

c(C) < 2LP < 20PT 

to prove an approximation guarantee of 2, However, the example proves that we will not be able to 
decrease this factor beyond 4/3. This follows from the fact that 

OPT < c(C) < aLP < aOPT => OPT/ LP < a 

then the best we can hope for is at most 4/3 by using this relaxation. This important property of 
the "bad examples" is captured in the concept of integrality gap. 

Definition 2 (Integrality gap) Given a relaxation LP(U) for an integer program IP(TT) that 
formulates a combinatorial (minimization) optimization problem on a collection of instances {II}, 
the integrality gap of the linear program relaxation is the largest ratio between the optimal solution 
of both formulations, namely: 

iptm 

Integrality gap = sup 

n L 1^(11) 

For the Vertex Cover LP relaxation, the integrality gap is exactly 2. To see that it is at least 
2, consider the complete graph G = K n , with unitary costs. The minimum vertex cover has cost 
71 — 1, while the linear program relaxation can assign 1/2 to all variables, which gives a total cost 
of n/2. Therefore, the integrality gap is at least -»■ 2. The upper bound follows from the 

2-approximation algorithm we will see in the next subsection. 

3.2 A 2-approximation Algorithm for Vertex Cover 

A natural approach to get an integral solution from a fractional solution is to round the fractional 
values. A simple rounding scheme for the vertex cover is as follows. 

1. Solve the linear programming relaxation given by (2a)- (2c), to get the fractional solution x* . 
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2. Compute the vertex cover as C = {v e V, x* > 1/2} (i.e., round each fractional variable to the 
nearest integer). 

Theorem 2 The above rounding scheme is a 2- approximation algorithm for the Vertex Cover prob- 
lem. 

Proof: First, we need to check that C is indeed a vertex cover. For each e = (u, v) € E, x* +x* > 1, 
so at least one of a;*, has value at least 1/2, and is in C. Next, the cost of this vertex cover 
satisfies 

c(C) = c " - 2 CvX *< = 2LP - 2 ° PT > 

v.x*>l/2 vEV 

hence the LP rounding is a 2-approximation algorithm for the vertex cover problem. □ 
This is a very basic (the simplest) example of rounding; more sophisticated rounding procedures 
have been used to design approximation algorithms; we'll see some in coming lectures. 

4 The Primal Dual Technique 

Yet another way of designing approximation algorithms for intractable problems is the primal dual 
method. The basic idea of the primal dual scheme is this: At every point of the algorithm, wc 
keep a feasible dual solution, and a corresponding infeasible integer primal solution. The dual 
variables are then modified at every step and so is the infeasible primal solution, so as to achieve 
primal feasibility. At this point, the dual gives a lower bound (for minimization problems) on the 
optimal primal objective function value, which is used to derive the approximation factor for the 
algorithm. The interesting thing about this technique is that we do not need to explicitly solve the 
linear program (as is the case in rounding); the linear program is used only in the analysis of the 
algorithm. 

We illustrate this method for the vertex cover problem. The linear program for the vertex cover 
problem is given by (2a)- (2c). The dual of this linear program is given by 

max^ y e 

S.t. Y V" ^ C v W ^ V > ( 3 ) 

y e > Ve e E. 

The primal dual algorithm for the vertex cover problem is as follows. In the algorithm, C 
corresponds to the set of vertices in the (supposed to be) vertex cover, and F is the set of edges in 
the graph not yet covered by C. 

1. y(v) ^0 VveV, C <- 0, F <- E. 

2. While F + 

3. Let e = (u, v) be any edge in F. 

4. Increase y e until the constraint (3) becomes tight for u or v. 

5. Add that corresponding vertex (say it is v) to C. 

6. F^F\5{v). 

Theorem 3 The above algorithm achieves an approximation ratio of 2 for the vertex cover problem. 
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Figure 3: Illustration of the primal-dual algorithm for the vertex cover problem. The cost of the 
vertices are indicated next to each vertex. Dotted edge denotes the edge currently under considera- 
tion, thick edges denote those already covered by the current vertex cover. The vertices in the cover 
arc shown as solid circles. 



Proof: First of all, it is clear that the set C returned by the algorithm is a vertex cover. Let 
y be the dual solution returned. Observe that by construction, this solution is dual feasible (we 
maintain dual feasibility throughout the execution). Furthermore, for any v £ C, we have that 
c v = J2e£8(v) Ve- Let us now compute the cost of the vertex cover returned by the algorithm. 

Yl Cv = ( y*) = Yl aeVe - 2 Y Ve 

v£C veC e£S(v) e£E eeE 

< 2LP (4a) 

< 20PT, (4b) 

where a e = 2, for edge e = (u, v) if both u, v G C, 1 otherwise. The inequality (4a) follows from 
weak duality, and inequality (4b) follows from the fact that the primal LP is a relaxation of the 
vertex cover problem. □ 
Figure 3 illustrates the execution of the primal-dual algorithm on a graph. For this instance, 
the algorithm returns a vertex cover of cost 9, whereas the optimal solution in this instance has 
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cost 7 (corresponding to the two vertices on the diagonal edge). The lower bound given by the dual 
solution has value 3 + 1 + 1 = 5. 
A few final remarks: 

• Dinur and Safra [2] have proved that it is NP-hard to approximate to the vertex cover with a 
factor better than 1.36. 

• Currently, there is no algorithm for the vertex cover problem which achieves an approximation 
ratio better than 2. So the two (simple!) algorithms presented here are, in fact, the present 
best known approximation algorithms for this problem. 

• Khot and Rcgev [3] have proved that it is UGC-hard to approximate vertex cover within a 
factor 2 — e, for any e > 0. 
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